Towards Accurate and Fast Evaluation of Multi-Stage Log-structured Designs

Multi-stage log-structured (MSLS) designs, such as Lev-elDB, RocksDB, HBase, and Cassandra, are a family ofstorage system designs that exploit the high sequentialwrite speeds of hard disks and flash drives by using mul-tiple append-only data structures. As a first step towardsaccurate and fast evaluation of MSLS, we propose newanalytic primitives and MSLS design models that quicklygive accurate performance estimates. Our model can al-most perfectly estimate the cost of inserts in LevelDB,whereas the conventional worst-case analysis gives 1.8–3.5X higher estimates than the actual cost. A few minutesof offline analysis using our model can find optimizedsystem parameters that decrease LevelDB’s insert cost byup to 9.4–26.2%; our analytic primitives and model alsosuggest changes to RocksDB that reduce its insert costby up to 32.0%, without reducing query performance orrequiring extra memory.